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ABSTRACT 

This paper contains summaries of the findings of 
research and meta analyses conducted on the effectiveness of 
computer-based instruction (CBI) on student achievement. Eight 
conclusions, based on the CBI research are then given: (1) when CBI 
and traditional instruction are compared, students receiving CBI 
demonstrate equal or better achievement; (2) equal or better 
achievement using CBI is obtained in less time; (3) the use of CBI 
improves student attitudes towards the use of computers in learning 
situations; (4) the positive effect on learning achievement occurs 
regardless of the type of CBI used, the type of computer system, or 
the age of the students; (5) teacher interaction proves to be 
effective in CBI situations; (6) there is little evidence to support 
the claim that learning to program will result in higher level 
cognitive skills and capabilities to learn; (7) tutorial and drill 
modes seem to be more effective for low-ability students than for 
middle- or high-ability students; and (8) the effect on learning 
achievement seems to be greatest for pre-college age learners. 
Research findings are also presented for social development factors 
and the cost effectiveness of CBI, and a summary statement and 
27-item bibliography conclude the document. (JB) 
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CBI Review 



The technological revolution of the past two decades has created a 
number of new information technologies , these include televison, video 
recording devices , laser discs, and microcomputers. Clearly, microcomputers 
are the dominant technology in education today. The microcomputer is the 
most versatile of the tec; iologies previoulsy listed. It has the 
distinction, when compared to other technologies, of being highly 
interactive. In this review, we will examine research on the effectiveness 
of computer-based instruction on student achievement. 



Effectiveness of Computer-Based Instruction 

Microcomputers are so new to the day-to-day aspects of education that a 
research base which provides some insight into the effectiveness of this new 
technology is limited. While we have a short history of using microcomputers 
in the classroom and very little empirical evidence to support our intuitions 
concerning how best to use this technology, there is a twenty-year history of 
experience with and research on the use of mainframe and mini-computers to 
instruct students. We will include here research on micro, mini, and 
main-frame computers. It seems resonable that much of what was learned in 
the past two decades from research on larger computers can be applied to 
today's microcomputers. 
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Evaluative Studies 

Soon after the introduction of computer-based instruction (CBI), 
researchers began to design studies which attempted to determine the 
effectiveness of CBI when compared to traditional instruction. Ia one of the 
first large-scale evaluative studies, Suppes and Morningstar (1963), compared 
the Stanford Achievement Test scores of 925 students who were given an 
average of ten minutes daily of CBI math computation instruction with those 
of 1028 control students who received traditional instruction. In four 
California schools across all elementary grade levels, CBI produced 
equivalent or superior results when effects were corrected for time spent in 
instruction. Students across all grade levels in an elementary school in 
Mississippi who received CBI instruction showed significant increases in 
their test performances when compared with control students. 

Atkinson (1968), conducted a similar comparison where 100 first graders 
received 20 minutes of daily computer based instruction in math. In this 
comparison, CBI was superior to the traditional instruction provided to a 
control group. In a later study of 50 matched pairs of male and female 
primary grade students, significant gains occurred only in the male group 
(Fletcher & Atkinson, 1972). 

The Educational Testing Service (Alderman, 1978; Murphy & Rhea-Appel, 
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1977) evaluated the implementation of two major comguter-based instructional 
systems in community colleges, PLATO (Programmed Logic for Automatic Teacher 
Operations) and TICCIT (Time-shared, Interactive, Computer-Controlled, 
Information Television). The two projects differ in several ways. PLATO is a 
large educational network supporting nearly 1000 terminals, each accessing 
content from a central library. Classroom teachers, coordinated by PLATO 
central staff, are involved in the preparation of the lessons. TICCIT, on 
the other hand, is a small, local computer-assisted instruction (CAI) 
facility using minicomputers and televison receivers. Teams of specialists, 
including teachers, produce the courseware. Both systems successfully 
produced advanced instructional systems capable of serving many students at 
one time. However, results of the evaluations are mixed. 

The PLATO evaluation covered five fields: accounting, biology, 
chemistry, English, and mathematics. Computer use in these fields 
represented supplemental or replacement instruciton for regular classroom 
work; in no cases were the PLATO programs in lieu of entire courses. The 
findings indicated a significant positive achievement effect for PLATO versus 
traditional classroom procedures in the area of mathematics. However, no 
further significant achievement effects were found for any other subjects, 
either in favor of PLATO or in favor of the traditional instruction (Murphy 
and Rhea-Appel, 1977). 

The TICCIT evaluation concerned both mathematics and English courses. 
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These applications represented entire courses, although the English TICCIT 
program included much more personal interaction between students and faculty 
than did the mathematics course. The results of the mathematics evaluation 
comparing the TICCIT courses to the traditional instruction indicated a 
significant achievement effect of TICCIT over the regular classroom, although 
fewer TICCIT students completed the course within the semester than did those 
of the regular classroom. Additionally, more students had favorable 
attitudes toward the lecture classes than towards the computer approach. 

The results of the English evaluation also indicated a significant 
achievement effect in favor ,of the TICCIT approach, and the completion rate 
for TICCIT students was the same as traditional instruction students. 
Additionally, there were not significant attitude differences in favor of 
either approach (Alderman, 1978). 

While the early evaluation studies produced some valuable information on 
the effects of computer-based instruction, the results from the studies were 
inconsistent and the conclusions drawn by the investigators were often 
unclear. In an attempt to gain a better understanding of the effect of CBI 
on achievement, reviews were written to bring the separately published 
studies together to reveal the common findings. These early reviews used a 
"box-score" technique for integrating the results. These box-score reviews 
generally reported the proportion of studies which were favorable and 
unfavorable toward CBI, as well as, narrative comments on the studies. 
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Box-Score Reviews 



In one of the first box-score reviews, Vinsonhaler and Bass (1972) 
summarized the results of ten major studies conducted from 1967 to 1970 
involving CBI drill and practice with more than 10,000 elementary school 
children from different sections of the country. The investigators concluded 
that children who received computerized drill and practice generally showed 
performance gains of 1 to 8 months over control children who received only 
traditional instruction. 



The Vinsonhaler and Bass conclusions were supported in a later review by 
Edwards, Norton, Taylor, Weiss, and Dusseldorp (1975) who evaluated the 
effects of drill and practice, problem-solving, simulation, and tutorial 
computer instruction programs for producing achievement gains in school 
children. Based upon results from six studies, they concluded that CBI plus 
traditional instruction was more effective than traditional instruction 
alone. CBI as a substitute for traditional instruction produced positive 
effects in nine studies and no difference compared with traditional 
instruction in eight others. Two studies in this review indicated that CBI 
drill and practice programs were more effective for low-ability students than 
for children functioning in the average range. These reviewers noted that 
CBI reduced the time it took students to learn, in addition, they concluded 
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that CBI produced better results than did traditional instruction on 
end-of -course examinations , but not on retention examinations. 

Jamison, Suppes, and Wells (1974) concluded that when CBI was used as a 
supplement to traditional instruction at the elementary level, achievement 
scores were improved, especially for disadvantaged students. At the 
secondary and college level, the investigators concluded that CBI was at 
least as effective as traditional instruction, and in some cases, CBI 
resulted in a substantial savings in student time. 

While the box-score reviews provided additional insight into the 
fundamental questions regarding the effectiveness of CBI, this type of review 
has certain limitations. For example, the box-score reviews do not say how 
much better one method is than another, they simply report how often the 
method comes out on top. Further, they do not use statistics to find the 
characteristics that distinguish studies with positive results from those 
with negative findings. In an attempt to overcome the limitations of 
box-score reviews, researchers have employed a more sophisticated method 
called meta-analysis. Meta-analysis, most simply, is an analyis of a large 
collection of results from individual studies for the purpose of integrating 
the findings. 
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Meta-Analyses 

Hartley (1977) was the first to apply meta-analysis to findings on CBI. 
She focused her analysis on math education in elementary and secondary 
schools. Hartley reported that the average effect of CBI was to raise 
student achievement from the 50th to the 66th percentile. Hartley also 
reported that the effects produced by CBI were not so large as those produced 
by programs of peer and cross-age tutoring, but they were far larger than 
effects produced by programmed instruction or individual learning packets. 

Burns and Bozeman (1981) also used meta-analysis to integrate findings 
on CBI in mathematics teaching in elementary and secondary schools. They 
found that computer-based tutorials raised achievement test results by .45 
standard deviations and that computer-based drill and practice raised test 
scores by .34 standard deviations. In a summary of their analysis, the 
reviewers wrote, "While no ultimate final answers related to CAI 
effectiveness or guarantors of success can be presented, the analysis and 
synthesis pf_ many studies do point to a significant enhancement of learning 
in instructional environments supplemented by_ CAI, at_ least in one curricular 
area - mathematics" (p. 37). 

Kulik, Kulik, and Cohen's (1980) review of research on CBI at the 
college level indicated that CBI had a modest positive effect for advanced 
learners. Their meta-analysis of 59 studies indicated that CBI produced 
gains of approximately .25 standard deviation units on content mastery exams 
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for college-level courses, an effect that was found uniformly across student 
ability level. Moreover, students tended to have a more positive attitude 
toward courses that included computer instruction. More importantly, in 
every study in which computer-based instruction was substituted for 
traditional teaching, the computer group completed the instruction in about 
two-thirds or less of the time required by traditional teaching methods. 

In a more recent meta-analysis, Kulik, Bangert, and Williams, (1983) 
analyzed 51 independent experimental studies which used CBI in grades six 
through twelve. Kulik et al. found that when CBI was used in instruction, 
student scores on final examinations were raised from the 50th to the 63rd 
percentile which represented a .32 standard deviation increase. In addition, 
Kulik and his colleagues reported that student attitudes toward the subject 
being learned and student ratings of the quality of instruction are slightly 
more favorable with CBI. Further, student's attitudes toward computers are 
significantly more positive as a result of CBI. 

Wise and Okey (1983) conducted a meta-analysis on twelve CBI studies 
which used microcomputers for the delivery of instruction. Eight of the 
twelve studies were done at the college level, two at the high school level, 
one in middle school and one at the pre-school level. Of some interest in 
these studies is the number of microcomputers available for the research. In 
nine of the twelve studies, one computer was available. In the remaining 
three studies, two, four, and ten microcomputers were made available for the 
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studies. These investigators reported that the mean effect size of .82 
standard deviation units on ten student outcomes was in favor of students 
receiving CBI. Unfortunately, the investigators did not describe the student 
outcome measures which made up the large effect size. 



A Longitudinal Study of CBI Effectiveness 

While the effectiveness of CBI has been demonstrated in a number of 
studies, few of these investigations have been conducted over long periods of 
time. For example, Wise and Okey (1983) report that in three of the studies 
they reviewed, the students had only one encounter with a computer and the 
greatest number of encounters for any study was only eleven. Thus, many of 
the findings being reported are based on brief encounters with CBI by the 
students. There is, however, one notable exception. The Educational Testing 
Service (ETS), in conjunction with the Los Angeles Unified School District 
(LAUSD) recently conducted a four-year study on the effectiveness of CBI for 
compensatory education students (Ragosta, Holland, & Jamison, 1982). The 
study was designed to answer questions about the effectiveness of three CBI 
currlculums — mathematics, reading, and language arts — when used for one 
year and over several years. 

Four elementary schools were each equipped with CBI labs using terminals 
and printers operated by a minicomputer. Half of the students in each school 
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attended the CBI lab for 10 to 20 minutes a day over four years. Each of the 
curriculums was designed to reinforce skills which students had already been 
taught in the classroom. The computer program adapted its delivery of each 
skill strand of the CBI curriculum to the performance level of each student 
and moved the student along at his/her own rate or progress. 

In this study, the learning effects were translated into equivalent 
percentile scores which allows comparison between CBI and traditional 
instruction; the fiftieth percentile being the average for the control 
students. On the curriculum specific tests designed for this study, the math 
CBI students showed significant and indisputable progress. They performed at 
the 79th percentile of their within-class groups at the end of only one year, 
at the 82nd percentile by the end of two years, and at the 89th percentile at 
the end of three years. 

While student gains using the reading and language arts CBI curriculums 
did not show the pattern of increasing performance over time that was evident 
with the mathematics curriculum, substantial gains (15 percentiles in reading 
and 26 percentiles in language arts) made during the first year were 
maintained over the three-year period. After the initial improvement, 
however, the gains were not substantial. 

Overall, each of the CBI curriculums demonstrated the capacity for 
improving the test scores of students requiring compensatory education as 

10 



CBI Review 



well as students performing at grade level (RAgosta, Holland, & Jamison, 
1982). The results from this study are consistent with other CBI studies. 
Dence (1980), and Gershman & Sakamoto (1981) have reported that low aptitude 
students profit more from the use of CBI than either average or high aptitude 
students* This seems to be especially true when the type of computer-based 
instruction is of the drill and practice or tutorial type. Further, many 
reluctant learners become active and interested learners when involved in 
computer supported programs. 



Conclusions 

Based upon the twenty years of CBI research, it seems clear that several 

conclusions can be drawn with regard to the effectiveness of CBI on student 
achievement: 

1. When CBI and traditional instruction are compared, students receiving CBI 

demonstrate equal or better achievement. 

2. When CBI and traditional instruction are compared, equal or better 

achievement using CBI is obtained in less time. 

3. The use of CBI improves student attitudes toward the use of computers in 

the learning situation. 

4. The positive effect on learning achievement occurs regardless of the type 

of CBI used, the type of computer system used, or the age range of 
students. 

5. "Primary CBI", where no teacher interaction occurs during the learning 
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process, is much less effective than "adjunct CBI" where teacher 
Interaction is a critical part of the instruction. 

6* While advocates of teaching computer programming claim that programming 
will result in higher level cognitive skills and capabilities to learn, 
there is little evidence on this matter, one way or the other. 

7 . Tutorial and drill modes seem to be more effective for low-ability 

students than for middle or high-ability students. 

8. The effect on learning achievement seems to be greatest for pre-college 

age learners. 

Social Development Factors 

Much has been written speculating on the effect that the microcomputer 
will have on the social development of students. The microcomputer is viewed 
by many as a machine which invites individual and sometimes asocial 
activity. To date, there is very little evidence which describes how this 
new technology will effect the social climate of classrooms. However, what 
little evidence there is suggests that microcomputers provide opportunities 
for interaction and collaboration among students in classrooms. Several 
recent research reports state that children often work together and teach 
their peers over microcomputers (Papert, Watt, diSessa, & Weir, 1979; 
Sheingold, Kane, Endreweit, & Billings, 1981). 

Hawkins, Sheingold, Gearhart, and Berger (1982) examined the type and 
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amount of social interaction that took place during regular classroom 
activities and around computers. The investigators observed the type and 
frequency of peer interactions that occurred in classroom work before and 
after computers were placed in the classroom. The findings indicate that 
more task-related interaction occurred among children when they were working 
with computers than when they were engaged in other non-teacher-directed 
classroom tasks (e.g., math and language activities). In addition, the 
results suggest that the computer may provide a context in classrooms where 
children recognize each other as helping resources. Thus, it would seem that 
the personal computer may actually enhance the social development of 
youngsters in the classroom rather than inhibit social development as some 
have suggested. 



The Cost of CBI 

Computer-Based Instruction research generally has shown that the use of 
the computer for delivering instruction results in positive student gains. 
Unfortunately, the cost of this delivery has been quite high. It is 
anticipated that the declining cost of today's microcomputer will bring the 
cost of CBI delivery within reasonable limits. This, however, remains to be 
seen. 

Cost estimates for CBI are highly variable and difficult to establish 
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with any degree of accuracy, particularly as CBI can be delivered on a 
variety of computer systems. However, in an attempt to provide prospective 
microcomputer users with some cost estimates, Pressman & Rosenbloom (1984) 
presented an example in which 15 microcomputers would be used in a school of 
750 to 1000 students. Costs were broken down into hardware and software and 
spread over a seven year period. While, seven years does not represent the 
maximum life of a microcomputer, this time period was chosen for illustrative 
purposes for estimating yearly costs. With an initial investment of $98,550, 
over seven years the adjusted average yearly costs would be about $17,360. 
When this figure is adjusted for yearly student use, Pressman and Rosenbloom 
estimate the average cost per student hour to be $1.32. 

In another cost projection, Melmed (1982) provides a detailed breakdown 
on the cost, in 1980 dollars, of equipping the nation's schools with enough 
microcomputers by 1990 so that each elementary and secondary school student 
could average 30 minutes per day at a computer. Melmed concludes, 



"For about $30 per student per year (including the costs of 
equipment, maintenance, courseware, and materials other than 
courseware), each student in the elementary and secondary 
education system could enjoy a computer-enriched instructional 
program averaging 30 minutes a day. This figure represents 1.2 
percent of the projected annual instructional budget of $2500 per 
student in 1990 (projected by the National Center for Education 
Statistics). If the cost of equipment were to be provided through 
a separate (capital) budget , the remaining annual operating cost 
of $5 would represent only 0.2 percent of the projected annual 
instructional budget spent for each student. This compares with 
about 0.8 percent of the instructional budget spent for all 
instructional materials in 1980. (p. 310) M 
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Summary 



Having reviewed a cross section of CBI research conducted over the past 
two decades, one can ask, what do we know about the effectiveness of 
computer-based instruction in education? First, there is evidence that 
computers can be used to effect positive student gains in all curricular 
areas, but especially in math. It also seems that CBI is especially powerful 
for disadvantaged and students with learning difficulties. It is clear that 
computers do not stifle the creative process, are not dehumanizing, and do 
not foster antisocial behavior or development. However, none of the 
potential benefits of CBI are inherent. On the contrary, the greatest gains 
from the use of the computer seem to occur when it is integrated thoughtfully 
into the on-going curriculum and not used as a replacement for existing 
courses* While CBI has reduced the dependence of instruction upon the 
quality of human effort to some extent, human effort and quality 
instructional materials still remain the major factor in the successful or 
unsuccessful use of computers in education. 
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